1. Field of the Invention
The present invention relates to a computer based browsing computer program product, system and method for browsing through documents stored in a computer. Specifically, the present invention facilitates the browsing of a document stored in a computer in such a way that it allows a rapid and holistic view of what contents are present and the organization of the material in the document, as well as rapid access to the items in the document involved. The document-browsing system in the present invention is also generalized to provide a novel human-computer interface, not just for browsing documents but also for interacting with and operating a computer to achieve the various functionalities normally carried out on the computer.
2. Description of the Related Art
Currently, the method for viewing documents stored in a computer is through a computer monitor screen. For documents that are longer/larger than can be contained within one screen, a way is provided to scroll the document up and down or to jump to a particular point in the document (through the use of, for example, a “mouse” coupled with scroll bars at the edges of the document displayed on the screen or through the use of hypertext links).
However, most people do not find this a particularly convenient way to view a document, as compared with a book. That this is true is evidenced in the fact that despite the trend of increased computer usage, there is not only no corresponding reduction in paper use, but an increase in paper quantity is required to print what is conveniently stored in a computer that can be seemingly easily and flexibly manipulated for viewing. Sometimes the printed version of the information involved is necessary, like in the case of printing a picture on a sheet of paper to be pasted on some surface (e.g., a wall), or in the case of the convenience of having a relatively light, hardy paperback book for reading in almost any possible situation and location. Reading textual material on the computer screen for a prolonged period of time also leads to visual fatigue. But there are times when a reader does not need to read for a prolonged period of time and is quite happy to sit at a desk in front of a desktop computer or handle a notebook computer away from the desk, and yet the reader would still prefer a printed version of the document in a hand-held format. This is especially true in the case of looking through manuals, including software manuals, to understand how to operate a device or software application. The irony is that computer software packages (included in computer readable media, such as optical disks, magnetic disks, semiconductor memory, etc.) usually come with thick and heavy manuals containing information which can easily be stored on a light-weight CD-ROM, and even if the manuals are sometimes dispensed in the form of a CD-ROM, more often than not people would print them out into a hard-copy format in order to facilitate reading.
What is involved when one interacts with the printed material in a book is a subtle and complicated process. To start with, material in a book is presented in a sequential order, with a continuity of material from page to page, and there is also a hierarchical structure in the material presented (as the material is organized into chapters, sections, subsections, etc.) because ideas in the material are related to each other in some kind of conceptual hierarchy. The human perceptual system inputs data in a sequential manner, and after a book is read from the beginning to the end in a sequential fashion, the brain then recreates the conceptual hierarchy after viewing the material involved. However, very often one does not read a book (or input the material involved) from the beginning to the end because (a) one wants to have an overview of the material present; (b) one is searching for something of interest to him/her; or (c) one is interested in reading only portions of the book (in the case of, say, reading the manual to understand how to operate something). In these cases, one browses through the subject book to find the material of unique interest to that reader.
Two basic things are achieved in the browsing process. First, the browser has a glimpse of what are the contents of the book document. Second, the browser has an idea of approximately where the items of interest are so that the browser can (a) return to look for them later when needed, and (b) have an understanding of the relationships between the material currently being viewed and other material (i.e., an understanding of the hierarchical structure involved). When browsing a book document, many finger-operations are required of the browser in order to flip through the pages and, together with the inherent sequential order imposed by the pages, very quickly allow the browser to have an understanding of the nature, location and organization of the material involved.
In the process of browsing through a book, one can perform the following operations:
(a) flip through the pages at varying speeds depending on the level of detail at which one wishes to view the material in the book;
(b) jump to the approximate location of the item of interest;
(c) change the direction of flipping (forward or backward) very rapidly because                (i) one would like to compare and contrast material on different pages,        (ii) after jumping to an approximate location of some items of interest one would like to find their exact locations, or        (iii) one is unsure of where the item of interest is and is in the process of searching for it; and        
(d) mark the locations of some pages of interest that one may want to later return.
All these operations are performed very rapidly with the fingers interacting with the flipping pages and with minimal unnecessary movements of the fingers and hands. Interestingly, a book/magazine with soft and flexible pages is harder to handle because more finger and hand movements are needed to browse through it, while books with stiff pages can be browsed with almost no movement of the hand.
In currently available methods of browsing through documents stored in a computer, e.g., the use of a mouse combined with scroll bars and buttons on the computer screen, more movements of the hands are necessary to effect the various operations described above. Also, fine control of the hand or fingers (depending on whether the mouse uses hand movement to move the cursor on the screen or finger movement like in the case of a track ball) is necessary to position the cursor on the screen at the required places. The process is both lengthy and clumsy. The lengthiness of the process taxes the human short term memory's ability to remember items encountered in the recent past for the purpose of establishing the relationships between items and the clumsiness of the process creates distraction and interferes with the short term memory process, a well known effect in perceptual psychology. These destroy the ability of the reader to form an idea of the contents and structure—i.e., a holistic view—of the document involved.
Because material in a book is organized into pages, it allows the reader to quantify approximately the amount of coverage given to each topic—e.g., three-quarters of a page, three-and-a-half pages, etc. In addition, page-oriented organization together with the fact that the pages are joined together in a sequential order with a continuity of material from page to page enhance the ability of the reader to better remember the location of various portions of the material involved (e.g., roughly how many pages from the pages currently in view). Also, unlike the process of scrolling through a document on a screen like what is normally done in a word-processor, wherein the contents become a blur and reading is impossible, when one moves through the material in a book through flipping, one is still able to read at least the approximate contents, if not the details. This, together with the page-oriented organization that enhances the user's impression of the location of material viewed, allow the user to form an impression of the organization of the material in the document.
Another method used for presenting information on the computer screen is flashing one page of information after another. This method eliminates the problem of blurred images but it also eliminates the continuity between different portions of the document. In turn, the lack of continuity eliminates the reader's ability to form impressions of the approximate absolute locations of these portions of the material involved and hence the impressions of the relationships between these portions of the material. As recognized by the present inventor, only the flipping method normally present in a book (e.g., a hardcover book) is able to provide both continuity and readable pages in view. It is due to these features that a person browsing through a book can acquire a good understanding of its contents, the location of specific items and organization of the material, and hence a holistic view of the book involved.
As recognized by the present inventor, it is because of the reasons set forth above that people still prefer to read a book in their hands rather than a document image displayed on a computer screen using currently available methods.
U.S. Pat. No. 5,467,102 (Kuno et al.) discloses a device for document viewing that consists of two display screens. One of the purposes of using two display screens is to allow the user to display two different pages from the document so that they can be read side-by-side (e.g., a diagram and its textual explanation). Another purpose is to allow a large picture to be displayed simultaneously on both pages. The Kuno et al. device allows users to change the speed of movement through the document through a pressure sensor—the more pressure applied, the faster the pages in the document are moved through. The Kuno et al. device also allows the document to be viewed in the forward or backward direction by pressing on a forward sensor area or a reverse sensor area respectively. One can also select a page to jump to by pressing on an icon displayed on the screen. However the Kuno et al. device still does not provide the same convenience as browsing through a book, primarily because when switching between the operations for different controls—the speed of movement through the document, the change of direction of viewing, and the jumping to different parts of the document—there are a lot more hand and finger movements than is the case in manipulating a physical book. Moreover, the Kuno et al. device is a specialized, relatively costly device with sensors and hardware built onto two display screens, whereas the present inventor recognizes that a lower cost and more practical device would be one that adds modularly to the existing computer system.
Currently, there are also computer mice that can eliminate the above-mentioned problem of positioning cursor on the computer screen with a conventional mouse (i.e., fine control of the hand or fingers is needed). These mice allow the user to specify “hot locations” on the screen on which the cursor “homes onto” with less fine control than conventional mice. Furthermore, a subset of these mice can generate “vertical only” or “horizontal only” movement of the cursor so that the scrolling process requires less fine control of the muscle than is required with a conventional mouse. These mice eliminate some, but not all, of the problems associated with the conventional methods of computer input as far as computer-based document browsing is concerned.
In U.S. Pat. No. 5,417,575 (1995) McTaggart discloses an electronic book that comprises laminated sheets bound together in the form of a book. On each of these sheets, printed material is arranged on the top layer and below that layer is an electronic backdrop containing thin light-emitting diodes (LED's) and pressure sensitive switches affixed onto a backing sheet. The LED's generate visual signals that can be seen through the top layer for the purpose of highlighting parts of the printed material. The pressure switches, positioned under certain items in the printed material, are for the purpose of sensing the user's selection of those items. A speaker is also provided on the book to generate audio signals for explaining the text or giving the user audio feedback. Contact or photo-sensitive switches are also embedded in the pages to allow the electronic circuits to know which pages are currently being viewed, so that the appropriate audio and visual signals can be generated. Even though this apparatus is in a form that allows a person to handle it like handling a typical book, with visual and audio enhancements of the printed material as well as facilities that accept the user's feedback, it is basically a hard-wired device that is not reprogrammable and different hardware has to be configured for books with different contents. No provision is available for downloading document files from a computer for display on the electronic book nor is the electronic book able to display any arbitrary document file. This device is hence not suitable for browsing through documents stored in a computer.
Therefore, the present inventor has identified there exists a need for a low-cost, modular device that can be connected to existing computer systems and that, through the computer executing a set of operations stored on a computer-readable medium, can effect rapidly the necessary browsing operations and together with a preferred mode of moving through a document—e.g., the flipping mode—that permits easy, effective computer-based document browsing that approaches that of browsing through a book. A feature of such a system is that it would exhibit functional attributes like those of a paper-book (i.e., a physical book such as a paper-back book).
The method of browsing a physical book can also benefit another aspect of human-computer interaction. In 1997, the most popular type of human-computer interface—the WINDOWS interface (e.g., MICROSOFT Corporation's WINDOWS 95® or APPLE Computer's MAC operating system, OS)—also suffers from many of the problems associated with reading documents on a computer. Basically, there are two situations when “windows” are created on the computer screen. The first is when the user needs to look into the contents of “directories” containing either other directories or computer programs and data files. In this situation, what are displayed in a window when it is opened are the contents of a directory. This way, a user can navigate among the directories organized in a tree structure and resident on the computer hard-disks to locate programs (often called “applications”) and data. The other is when a user activates a program, and the program either generates some output to be displayed on the computer screen or is waiting to accept some user input (usually either a command to the program or data to be fed into the program). In the WINDOWS interface, these output and input are presented and accepted through a “window”, referred to as the working window of a program.
In the case of the use of multiple windows to display the contents of directories in the process of searching for a certain item (either a directory, a program or a data file), if one knows exactly under which directory an item of interest is placed, the tree-oriented structure present in this kind of system can greatly facilitate the search for the item. However, if one does not know where the item is placed, one would need to carry out a large number of operations opening/entering and closing/exiting directories, often across and up and down many levels of the directory structure, and this is carried out with a large number of, and often tiring, operations involving clicking and dragging the mouse (or other pointing device).
The situation is compounded by the fact that often the contents of a directory cannot fit entirely within a window. When that happens, a mouse-cum-scroll-bar method, much like that used for browsing through documents on a computer, is available for use to reveal the hidden parts of the directory involved. The situation is similar to viewing documents on computers, where the process of entering, exiting and scrolling up and down directory windows to search for items of interest demands a large number of finger and hand operations and movement. Also, the flashing in and out of view of these directory windows results in a discrete, non-continuous, and random sequence of images. This results in a poor idea of items' location in the entire directory structure as well as the inability to form a holistic view of the entire collection of program and data files resident on the computer's hard-disk.
In the case of computer programs using multiple windows to interact with a user, because each opened window tends to occlude other windows already present, it is often the case that in the process of accomplishing a certain task (e.g., creating a multimedia document), a large and often confusing sequence of windows has to be opened, closed or moved aside, each window representing an activation of a program to accomplish some subtasks (e.g., extracting some text from a text file, extracting a sub-part of an image from an image file, carrying out some processing on the images thus obtained, “pasting” all these items together in yet another program, etc.) Again a large number of finger and hand operations and movement is required to carry out these operations through the use of a mouse. Furthermore, because the various windows thus opened in this process do not have any fixed locations, and their relative locations keep changing, an excessive amount of mouse operations is often needed just to locate these opened windows or put them away for later use.
The WINDOWS interface derives from the desktop metaphor, in which the computer screen is likened to the desktop, and the windows are thus analogous to items that one normally handles on a desktop. While the desktop interface is a large improvement over the previous kind of command line interface (e.g., MICROSOFT's DOS®), primarily because of its user-friendly graphical interface and menu driven input, it suffers from the same problem of the desktop: when the items to be handled are many, the desktop tends to become very messy, and one often has to keep pushing aside many items to look for other items.
As a practical matter, when a large number of sheets of papers are handled by a person, either to collect information on various things or to scribble down ideas, one often joins (perhaps by a staple or binder clip) these sheets of paper into a temporary “book”, or one uses a scrap book with blank pages on which the user may scribble-down ideas. However, the working items, once bound together, are not necessarily related to one another, and thus, are different than a paper book in which the various portions of material are often related to one another. This organization of these sheets of paper allows one to obtain the usual advantages associated with a book, namely, the items of interest acquire absolute locations, and together with the finger/hand operations one can normally apply to a book to search for information, the items of interest can easily be located. Because the pages thus organized in a book can be quickly and easily located and viewed, the effect is tantamount to having multiple pages in view almost simultaneously, and hence it will alleviate the problems associated with the occlusion of pages by other pages or windows by other windows.
Thus, the present inventor has identified that there is a need for a higher level of organization built onto the currently available WINDOWS interface—a “Books” interface—that can be used in conjunction with the above-mentioned low-cost, modular device that can alleviate many of the problems associated with the WINDOWS interface in order to facilitate human-computer interaction.